Systems and methods for managing voice queries using pronunciation information

ABSTRACT

The system receives a voice query at an audio interface and converts the voice query to text. The system can determine pronunciation information during conversion and generate metadata that indicates a pronunciation of one or more words of the query, include phonetic information in the text query, or both. A query includes one or more entities that may be more accurately identified based on pronunciation. The system searches for information, content, or both among one or more databases based on the generated text query, pronunciation information, user profile information, search histories or trends, and optionally other information. The system identifies one or more entities or content items that match the text query, and retrieves the identified information to provide to the user.

BACKGROUND

The present disclosure relates to systems for managing voice queries,and, more particularly, systems for managing voice queries based onpronunciation information.

SUMMARY

In a conversation system, when a user fires a voice query to the system,the speech is converted to text using an Automatic Speech Recognition(ASR) module. This text then forms the input to a conversation system,which determines a response to the text. For example, when a user says“show me Tom Cruise movies” then the ASR module converts the users voiceto text and fires it to the conversation system. The conversation systemonly acts on the text it receives on from the ASR module. Sometimes inthis process the conversation system loses the pronunciation details ofwords or sounds included in the user's query. The pronunciation detailsmay provide information that can help with the search, especially whenthe same word has more than one pronunciation, and the pronunciationscorrespond to different meanings.

The present disclosure describes systems and methods that perform asearch based on multiple contextual inputs to predict a user's intendedsearch query as the user speaks the query words. The search may be basedon multiple contextual inputs that include, for example, user searchhistory, user likes and dislikes, general trends, pronunciation detailsof the query words, and any other suitable information. The applicationreceives a voice query and generates a text query that is representativeof the voice query. The application uses pronunciation information,which may be included in the text query, included in metadata associatedwith the text query, or included in metadata of entities in a databaseto more accurately retrieve search results.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the disclosure will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows a block diagram of an illustrative system for generating atext query, in accordance with some embodiments of the presentdisclosure;

FIG. 2 shows a block diagram of an illustrative system for retrievingcontent in response to a voice query, in accordance with someembodiments of the present disclosure;

FIG. 3 shows a block diagram of an illustrative system for generatingpronunciation information, in accordance with some embodiments of thepresent disclosure;

FIG. 4 is a block diagram of an illustrative user device, in accordancewith some embodiments of the present disclosure;

FIG. 5 shows a block diagram of an illustrative system for responding toa voice query, in accordance with some embodiments of the presentdisclosure;

FIG. 6 shows a flowchart of an illustrative process for responding to avoice query based on pronunciation information, in accordance with someembodiments of the present disclosure;

FIG. 7 shows a flowchart of an illustrative process for responding to avoice query based on alternate representations, in accordance with someembodiments of the present disclosure;

FIG. 8 shows a flowchart of an illustrative process for generatingmetadata for an entity based on pronunciation, in accordance with someembodiments of the present disclosure; and

FIG. 9 shows a flowchart of an illustrative process for retrievingcontent associated with an entity of a voice query, in accordance withsome embodiments of the present disclosure.

DETAILED DESCRIPTION

In some embodiments, the present disclosure is directed to a systemconfigured to receive a voice query from a user, analyze the voicequery, and generate a text query (e.g., the translation) for searchingfor content or information. The system responds to the voice query basedin part on pronunciation of one or more keywords. For example, in theEnglish language, there are multiple words having the same spelling butdifferent pronunciations. This may be especially true with the names ofpeople. Some examples include:

Name Pronunciation 1 Pronunciation 2 LOUIS loo-ee loo-ihs LEAH lee-uhlay-uh HARRY 'hæri 'heri ANTHONY ant-uhnee anth-uhneeTo illustrate, a user may voice “Show me the interview of Louis” to anaudio interface of the system. The system may generate the illustrativetext queries such as:

-   -   Option 1) “Show me the interview of Louis Freeh with Fraud        Magazine”    -   Option 2) “Show me the interview of Lewis Black that aired on        CBS”        The resulting text query depends on how the user has spoken the        word “Louis.” If user pronounced “LOO-ee” then the system        selects option 1 or applies more weighting to option 1. If user        pronounced “LOO-his” then the system selects option 2 or applies        more weighting to option 2. Without pronunciation taken into        account, the system likely would not be able to accurately        respond to the voice query.

In some circumstances, a voice query that includes a partial name of apersonality may cause ambiguity in detecting that person correctly(e.g., referred to as a “non-definitive personality search query”). Forexample, if the user voices “Show me movies with Tom” or “Show me theinterview of Louis,” then the system will have to determine which Tom orLouis/Louie/Lewis the user is asking about. In addition to pronunciationinformation, the system may analyze one or more contextual inputs suchas, for example, user search history (e.g., previous queries and searchresults), user likes/dislikes/preferences (e.g., from a user's profileinformation), general trends (e.g., of a plurality of users), popularity(e.g., among a plurality of users), any other suitable information, orany combination thereof. The system retains the pronunciationinformation in a suitable form (e.g., in the text query itself, or inmetadata associated with the text query) such that it is not lost afterthe automatic speech recognition (ASR) process.

In some embodiments, for pronunciation information to be used by thesystem, the information field among which the system searches mustinclude pronunciation information for comparison to the query. Forexample, the information field may include information about entitiesthat include pronunciation metadata. The system may perform a phonetictranslation process, which takes the user's voice query as input andtranslates it to text, which when read back, sounds phoneticallycorrect. The system may be configured to use the output of the phonetictranslation process and pronunciation metadata to determine searchresults. In an illustrative example, pronunciation metadata stored foran entity may include:

ID: WIKI04556 NAME: Louis Farrakhan PRONUNCIATION: LOO-his FAH-rah-kahn

In some embodiments, the present disclosure is directed to a systemconfigured to receive a voice query from a user, analyze the voicequery, and generate a text query (e.g., the translation) for searchingfor content or information. The information field among which the systemsearches includes pronunciation metadata, alternative textrepresentations of an entity, or both. For example, a user fires a voicequery to the system, and the system first converts the voice to textusing an ASR module. The resulting text then forms the input to aconversation system (e.g., that performs action in response to thequery). To illustrate, if the user says “show me Tom Cruise movies,”then the ASR module converts the user's speech to text and fires thetext query to the conversation system. If the entity corresponding to“Tom Cruise” is present in the data, the system matches it with the text‘Tom Cruise’ and returns appropriate results (e.g., information aboutTom Cruise, content featuring Tom Cruise or content identifiersthereof). When an entity is present in data (e.g., of the informationfield) and can be accessed directly using the entity title, the entitymay be referred to as being “reachable.” Reachability is of primeimportance for systems performing search operations. For example, ifsome data (e.g., a movie, artist, television series, or other entity) ispresent in the system, and associated data stored, but the user can'taccess that information, then the entity may be termed “unreachable.”Unreachable entities in a data system represent a failure of the searchsystem.

FIG. 1 shows a block diagram of illustrative system 100 for generating atext query, in accordance with some embodiments of the presentdisclosure. System 100 includes ASR module 110, conversation system 120,pronunciation metadata 150, user profile information 160 and one or moredatabases 170. For example, ASR module 110 and conversation system 120,which together may be included in system 199, may be used to implement aquery application.

A user may voice query 101 which includes speech “Show me that Louisinterview from last week” to an audio interface of system 199. ASRmodule 110 is configured to sample, condition, and digitize the receivedaudio input and analyze the resulting audio file to generate a textquery. In some embodiments, ASR module 110 retrieves information fromuser profile information 160 to help generate the text query. Forexample, voice recognition information for the user may be stored inuser profile information 160, and ASR module 110 may use voicerecognition information to identify the speaking user. In a furtherexample, system 199 may include user profile information 160, stored insuitable memory. ASR module 110 may determine pronunciation informationfor the voiced word “Louis.” Because there are more than onepronunciation for the text word “Louis,” system 199 generates the textquery based on the pronunciation information. Further, the sound“Loo-his” can be converted to text as “Louis” or “Lewis,” andaccordingly contextual information may help in identifying the correctentity of the voice query (e.g., Lewis as in Lewis Black, as opposed toLouis as in Louis Farrakhan). In some embodiments, conversation system120 is configured to generate the text query, respond to the text query,or both, based on the recognized words from ASR module 110, contextualinformation, user profile information 160, pronunciation metadata 150,one or more databases 170, any other information, or any combinationthereof. For example, conversation system 120 may generate a text queryand then compare with text query with pronunciation metadata 150 for aplurality of entities to determine a match. In a further example,conversation system 120 may compare one or more recognized words topronunciation metadata 150 for a plurality of entities to determine amatch and then generate the text query based on the identified entity.In some embodiments, conversation system 120 generates a text query withaccompanying pronunciation information. In some embodiments,conversation system 120 generates a text query with embeddedpronunciation information. For example, the text query may include aphonetic representation of a word such as “loo-ee” rather than a correctgrammatical representation “Louis.” In a further example, pronunciationmetadata 150 may include one or more reference phonetic representationswith which the text query may be compared.

User profile information 160 may include user identification information(e.g., name, an identifier, address, contact information), user searchhistory (e.g., previous voice queries, previous text queries, previoussearch results, feedback on previous search results or queries), userpreferences (e.g., search settings, favorite entities, keywords includedin more than one query), user likes/dislikes (e.g., entities followed bya user in a social media application, user inputted information), otherusers connected to the user (e.g., friends, family members, contacts ina social networking application, contacts stored in a user device), uservoice data (e.g., audio samples, signatures, speech patterns, or filesfor identifying the user's voice), any other suitable information abouta user, or any combination thereof.

One or more databases 170 include any suitable information forgenerating a text query, responding to a text query, or both. In someembodiments, pronunciation metadata 150, user profile information 160,or both may be included in one or more databases 170. In someembodiments, one or more databases 170 include statistical informationfor a plurality of users (e.g., search histories, content consumptionhistories, consumption patterns). In some embodiments, one or moredatabases 170 include information about a plurality of entitiesincluding persons, places, objects, events, content items, media contentassociated with one or more entities, or a combination thereof.

FIG. 2 shows a block diagram of illustrative system 200 for retrievingcontent in response to a voice query, in accordance with someembodiments of the present disclosure. System 200 includes speechprocessing system 210, search engine 220, entity database 250, and userprofile information 240. Speech processing system 210 may identify anaudio file and may analyze the audio file for phonemes, patterns, words,or other elements from which keywords may be identified. In someembodiments, speech processing system 210 may analyze an audio input inthe time domain, spectral domain, or both to identify words. Forexample, speech processing system 210 may analyze the audio input in thetime domain to determine periods of time during which speech occurs(e.g., to eliminate pauses or periods of silence). Speech processingsystem 210 may then analyze each period of time in the spectral domainto identify phonemes, patterns, words, or other elements from whichkeywords may be identified. Speech processing system 210 may output agenerated text query, one or more words, pronunciation information, or acombination thereof. In some embodiments, speech processing system 210may retrieve data from user profile information 240 for voicerecognition, speech recognition, or both.

Search engine 220 receives the output from speech processing system 210,and, in combination with search settings 221 and context information222, generates a response to a text query. Search engine 220 may useuser profile information 240 to generate, modify, or respond to a textquery. Search engine 220 searches among data of database of entities 250using the text query. Database of entities 250 may include metadataassociated with a plurality of entities, content associated with theplurality of entities, or both. For example, data may include anidentifier for an entity, details describing an entity, a titlereferring to the entity (e.g., which may include a phoneticrepresentation or alternative representation), phrases associated withthe entity (e.g., which may include a phonetic representation oralternative representation), links (e.g., IP addresses, URLs, hardwareaddresses) associated with the entity, keywords associated with theentity (e.g., which may include a phonetic representation or alternativerepresentation), any other suitable information associated with anentity, or any combination thereof. When search engine 220 identifiesone or more entities that match keywords of the text query, identifiesone or more content items that match keywords of the text query, orboth, search engine 220 may then provide information, content, or bothto the user as response 270 to the text query. In some embodiments,search settings 221 include which databases, entities, types ofentities, types of content, other search criteria, or any combinationthereof to affect the generation of the text query, the retrieval of thesearch results, or both. In some embodiments, context information 222includes genre information (e.g., to further narrow the search field),keywords, database identification (e.g., which database likely includesthe target information or content), types of content (e.g., by date,genre, title, format), any other suitable information, or anycombination thereof. Response 270 may include, for example, content(e.g., a displayed video), information, a listing of search results,links to content, any other suitable search results, or any combinationthereof.

FIG. 3 shows a block diagram of illustrative system 300 for generatingpronunciation information, in accordance with some embodiments of thepresent disclosure. System 300 includes text-to-speech engine 310 andspeech-to-text engine 320. In some embodiments, system 300 determinespronunciation information independent of a text or voice query. Forexample, system 300 may generate metadata for one or more entities(e.g., such as pronunciation metadata 150 of system 100, or metadatastored in database of entities 250 of system 200). Text-to-speech engine310 may identify a first text string 302, which may include an entityname or other identifier that is likely to be included in a voice query.For example, text-to-speech engine 310 may identify a “name” field ofentity metadata rather than an “ID” field, since a user is more likelyto speak a voice query including a name rather than a numeric oralphanumeric identifier (e.g., the user speaks “Louis” rather than“WIKI04556”). Text-to-speech engine 310 generates audio output 312, at aspeaker or other audio device, based on the first text string. Forexample, text-to-speech engine 310 may use one or more settings tospecify voice details (e.g., male/female voice, accents, or otherdetails), playback speed, or any other suitable settings that may affectthe generated audio output. Speech-to-text engine 320 receives an audioinput 313 at a microphone or other suitable device from audio output 312(e.g., in addition to or in place of an audio file that may be stored),and generates a text conversion of audio input 313 (e.g., in addition toor in place of storing an audio file of the recorded audio).Speech-to-text engine 320 may use processing settings to generate a newtext string 322. New text string 322 is compared with first text string302. If new text string 322 is identical to text string 302, then nometadata need be generated because a voice query may result inconversion to an accurate text query. If new text string 322 is notidentical to text string 302, then this indicates that a voice querymight be incorrectly converted to a text query. Accordingly, if new textstring 322 is not identical to text string 302, then speech-to-textengine 320 includes new text string 322 in metadata associated with theentity that text string 302 is associated with. System 300 may identifya plurality of entities, and for each entity, generate metadata thatincludes resulting text strings (e.g., such as new text string 322) fromtext-to-speech engine 310 and speech-to-text engine 320. In someembodiments, for a given entity, text-to-speech engine 310,speech-to-text engine 320, or both may use more than one setting togenerate more than one new text string. Accordingly, since the more thanone text strings are different from text string 302, then each new textstring may be stored in the metadata. For example, differentpronunciations or interpretations of pronunciations arising fromdifferent settings may generate different new text strings, which may bestored in preparation for voice queries from different users. Bygenerating and storing alternative representations (e.g., text string302 and new text string 322), system 300 may update metadata to allowmore accurate searching (e.g., improve the reachability of entities, andthe accuracy of searching).

In an illustrative example, for an entity, system 300 may identify thetitle and related phrases, pass each phrase to text-to-speech engine 310and save the respective audio files, and then pass each respective audiofile to speech-to-text engine 320 to get an ASR transcript (e.g., newtext string 322). If the ASR transcript is different from the originalphrase (e.g., text string 302), system 300 adds the ASR transcript tothe related phrases of the entity (e.g., as stored in the metadata). Insome embodiments, system 300 does not require any manual work, and maybe fully automated (e.g., no user input is required). In someembodiments, when a user fires a query and does not get the desiredresult, system 300 is alerted. In response, a person manually identifieswhat should have been the correct entity for the query. The incorrectresult is stored and provides information for future queries. System 300addresses the potential inaccuracy at the metadata level rather than thesystem level. The analysis of text strings 302 for many entities may beexhaustive and automatic, so that all wrong cases are identifiedbeforehand (e.g., prior to a user's voice query) and are resolved.System 300 does not require a user to provide the voice query togenerate a wrong case (e.g., an alternative representation). System 300may be used to emulate a user's interaction with a query system toforecast potential sources of error in performing searches.

A user may access content, an application (e.g., for interpreting avoice query), and other features from one or more of their devices(i.e., user equipment or audio equipment), one or more network-connecteddevices, one or more electronic devices having a display, or acombination thereof, for example. Any of the illustrative techniques ofthe present disclosure may be implemented by a user device, a deviceproviding a display to a user, or any other suitable control circuitryconfigured to respond to a voice query and generate a display content toa user.

FIG. 4 shows generalized embodiments of an illustrative user device.User equipment system 401 may include set-top box 416 that includes, oris communicatively coupled to, display 412, audio equipment 414, anduser input interface 410. In some embodiments, display 412 may include atelevision display or a computer display. In some embodiments, userinput interface 410 is a remote-control device. Set-top box 416 mayinclude one or more circuit boards. In some embodiments, the one or morecircuit boards include processing circuitry, control circuitry, andstorage (e.g., RAM, ROM, hard disk, removable disk, etc.). In someembodiments, circuit boards include an input/output path. Each one ofuser equipment device 400 and user equipment system 401 may receivecontent and data via input/output (hereinafter “I/O”) path 402. I/O path402 may provide content and data to control circuitry 404, whichincludes processing circuitry 406 and storage 408. Control circuitry 404may be used to send and receive commands, requests, and other suitabledata using I/O path 402. I/O path 402 may connect control circuitry 404(and specifically processing circuitry 406) to one or morecommunications paths (described below). I/O functions may be provided byone or more of these communications paths but are shown as a single pathin FIG. 4 to avoid overcomplicating the drawing. While set-top box 416is shown in FIG. 4 for illustration, any suitable computing devicehaving processing circuitry, control circuitry, and storage may be usedin accordance with the present disclosure. For example, set-top box 416may be replaced by, or complemented by, a personal computer (e.g., anotebook, a laptop, a desktop), a network-based server hosting auser-accessible client device, a non-user-owned device, any othersuitable device, or any combination thereof.

Control circuitry 404 may be based on any suitable processing circuitrysuch as processing circuitry 406. As referred to herein, processingcircuitry should be understood to mean circuitry based on one or moremicroprocessors, microcontrollers, digital signal processors,programmable logic devices, field-programmable gate arrays (FPGAs),application-specific integrated circuits (ASICs), etc., and may includea multi-core processor (e.g., dual-core, quad-core, hexa-core, or anysuitable number of cores) or supercomputer. In some embodiments,processing circuitry is distributed across multiple separate processorsor processing units, for example, multiple of the same type ofprocessing units (e.g., two Intel Core i7 processors) or multipledifferent processors (e.g., an Intel Core i5 processor and an Intel Corei7 processor). In some embodiments, control circuitry 404 executesinstructions for an application stored in memory (e.g., storage 408).Specifically, control circuitry 404 may be instructed by the applicationto perform the functions discussed above and below. For example, theapplication may provide instructions to control circuitry 404 togenerate the media guidance displays. In some implementations, anyaction performed by control circuitry 404 may be based on instructionsreceived from the application.

In some client/server-based embodiments, control circuitry 404 includescommunications circuitry suitable for communicating with an applicationserver or other networks or servers. The instructions for carrying outthe above-mentioned functionality may be stored on the applicationserver. Communications circuitry may include a cable modem, anintegrated-services digital network (ISDN) modem, a digital subscriberline (DSL) modem, a telephone modem, an ethernet card, or a wirelessmodem for communications with other equipment, or any other suitablecommunications circuitry. Such communications may involve the Internetor any other suitable communications networks or paths. In addition,communications circuitry may include circuitry that enables peer-to-peercommunication of user equipment devices, or communication of userequipment devices in locations remote from each other (described in moredetail below).

Memory may be an electronic storage device such as storage 408 that ispart of control circuitry 404. As referred to herein, the phrase“electronic storage device” or “storage device” should be understood tomean any device for storing electronic data, computer software, orfirmware, such as random-access memory, read-only memory, hard drives,optical drives, solid state devices, quantum storage devices, gamingconsoles, gaming media, any other suitable fixed or removable storagedevices, and/or any combination of the same. Storage 408 may be used tostore various types of content described herein as well as mediaguidance data described above. Nonvolatile memory may also be used(e.g., to launch a boot-up routine and other instructions). Cloud-basedstorage, for example, may be used to supplement storage 408 or insteadof storage 408.

A user may send instructions to control circuitry 404 using user inputinterface 410. User input interface 410, display 412, or both mayinclude a touchscreen configured to provide a display and receive hapticinput. For example, the touchscreen may be configured to receive hapticinput from a finger, a stylus, or both. In some embodiments, equipmentdevice 400 may include a front-facing screen and a rear-facing screen,multiple front screens, or multiple angled screens. In some embodiments,user input interface 410 includes a remote-control device having one ormore microphones, buttons, keypads, any other components configured toreceive user input, or combinations thereof. For example, user inputinterface 410 may include a handheld remote-control device having analphanumeric keypad and option buttons. In a further example, user inputinterface 410 may include a handheld remote-control device having amicrophone and control circuitry configured to receive and identifyvoice commands and transmit information to set-top box 416.

Audio equipment 414 may be provided as integrated with other elements ofeach one of user device 400 and user equipment system 401 or may bestand-alone units. The audio component of videos and other contentdisplayed on display 412 may be played through speakers of audioequipment 414. In some embodiments, the audio may be distributed to areceiver (not shown), which processes and outputs the audio via speakersof audio equipment 414. In some embodiments, for example, controlcircuitry 404 is configured to provide audio cues to a user, or otheraudio feedback to a user, using speakers of audio equipment 414. Audioequipment 414 may include a microphone configured to receive audio inputsuch as voice commands and speech (e.g., including voice queries). Forexample, a user may speak letters or words that are received by themicrophone and converted to text by control circuitry 404. In a furtherexample, a user may voice commands that are received by the microphoneand recognized by control circuitry 404.

An application (e.g., for managing voice queries) may be implementedusing any suitable architecture. For example, a stand-alone applicationmay be wholly implemented on each one of user device 400 and userequipment system 401. In some such embodiments, instructions for theapplication are stored locally (e.g., in storage 408), and data for useby the application is downloaded on a periodic basis (e.g., from anout-of-band feed, from an Internet resource, or using another suitableapproach). Control circuitry 404 may retrieve instructions for theapplication from storage 408 and process the instructions to generateany of the displays discussed herein. Based on the processedinstructions, control circuitry 404 may determine what action to performwhen input is received from input interface 410. For example, movementof a cursor on a display up/down may be indicated by the processedinstructions when input interface 410 indicates that an up/down buttonwas selected. An application and/or any instructions for performing anyof the embodiments discussed herein may be encoded on computer-readablemedia. Computer-readable media includes any media capable of storingdata. The computer-readable media may be transitory, including, but notlimited to, propagating electrical or electromagnetic signals, or may benon-transitory including, but not limited to, volatile and non-volatilecomputer memory or storage devices such as a hard disk, floppy disk, USBdrive, DVD, CD, media card, register memory, processor cache, RandomAccess Memory (RAM), etc.

In some embodiments, the application is a client/server-basedapplication. Data for use by a thick or thin client implemented on eachone of user device 400 and user equipment system 401 is retrieved ondemand by issuing requests to a server remote from each one of userequipment device 400 and user equipment system 401. For example, theremote server may store the instructions for the application in astorage device. The remote server may process the stored instructionsusing circuitry (e.g., control circuitry 404) and generate the displaysdiscussed above and below. The client device may receive the displaysgenerated by the remote server and may display the content of thedisplays locally on user device 400. This way, the processing of theinstructions is performed remotely by the server while the resultingdisplays, which may include text, a keyboard, or other visuals, areprovided locally on user device 400. User device 400 may receive inputsfrom the user via input interface 410 and transmit those inputs to theremote server for processing and generating the corresponding displays.For example, user device 400 may transmit a communication to the remoteserver indicating that an up/down button was selected via inputinterface 410. The remote server may process instructions in accordancewith that input and generate a display of the application correspondingto the input (e.g., a display that moves a cursor up/down). Thegenerated display is then transmitted to user device 400 forpresentation to the user.

In some embodiments, the application is downloaded and interpreted orotherwise run by an interpreter or virtual machine (e.g., run by controlcircuitry 404). In some embodiments, the application may be encoded inthe ETV Binary Interchange Format (EBIF), received by control circuitry404 as part of a suitable feed, and interpreted by a user agent runningon control circuitry 404. For example, the application may be an EBIFapplication. In some embodiments, the application may be defined by aseries of JAVA-based files that are received and run by a local virtualmachine or other suitable middleware executed by control circuitry 404.

FIG. 5 shows a block diagram of illustrative network arrangement 500 forresponding to a voice query, in accordance with some embodiments of thepresent disclosure. Illustrative system 500 may be representative ofcircumstances in which a user provides a voice query at user device 550,views content on a display of user device 550, or both. In system 500,there may be more than one type of user device, but only one is shown inFIG. 5 to avoid overcomplicating the drawing. In addition, each user mayutilize more than one type of user device and also more than one of eachtype of user device. User device 550 may be the same as user device 400of FIG. 4, user equipment system 401, any other suitable device, or anycombination thereof.

User device 550, illustrated as a wireless-enabled device, may becoupled to communications network 510 (e.g., connected to the Internet).For example, user device 550 is coupled to communications network 510via a communications path (e.g., which may include an access point). Insome embodiments, user device 550 may be a computing device coupled tocommunications network 510 via a wired connection. For example, userdevice 550 may also include wired connections to a LAN, or any othersuitable communications link to network 510. Communications network 510may be one or more networks including the Internet, a mobile phonenetwork, mobile voice or data network (e.g., a 4G or LTE network), cablenetwork, public switched telephone network, or other types ofcommunications network or combinations of communications networks.Communications paths may include one or more communications paths, suchas a satellite path, a fiber-optic path, a cable path, a path thatsupports Internet communications, free-space connections (e.g., forbroadcast or other wireless signals), or any other suitable wired orwireless communications path or combination of such paths. Althoughcommunications paths are not drawn between user device 550 and networkdevice 520, these devices may communicate directly with each other viacommunications paths, such as those described above, as well as othershort-range point-to-point communications paths, such as USB cables,IEEE 1394 cables, wireless paths (e.g., Bluetooth, infrared, IEEE802-11x, etc.), or other short-range communication via wired or wirelesspaths. BLUETOOTH is a certification mark owned by Bluetooth SIG, INC.Devices may also communicate with each other directly through anindirect path via communications network 510.

System 500, as illustrated, includes network device 520 (e.g., a serveror other suitable computing device) coupled to communications network510 via a suitable communications path. Communications between networkdevice 520 and user device 550 may be exchanged over one or morecommunications paths but are shown as a single path in FIG. 5 to avoidovercomplicating the drawing. Network device 520 may include a database,one or more applications (e.g., as an application server, host server).A plurality of network entities may exist and be in communication withnetwork 510, but only one is shown in FIG. 5 to avoid overcomplicatingthe drawing. In some embodiments, network device 520 may include onesource device. In some embodiments, network device 520 implements anapplication that communicates with instances of applications at manyuser devices (e.g., user device 550). For example, an instance of asocial media application may be implemented on user device 550, withapplication information being communicated to and from network device520, which may store profile information for the user (e.g., so that acurrent social media feed is available on other devices than user device550). In a further example, an instance of a search application may beimplemented on user device 550, with application information beingcommunication to and from network device 520, which may store profileinformation for the user, search histories from a plurality of users,entity information (e.g., content and metadata), any other suitableinformation, or any combination thereof.

In some embodiments, network device 520 includes one or more types ofstored information, including, for example, entity information,metadata, content, historical communications and search records, userpreferences, user profile information, any other suitable information,or any combination thereof. Network device 520 may include anapplications-hosting database or server, plug-ins, a software developerskit (SDK), an applications programming interface (API), or othersoftware tools configured to provide software (e.g., as downloaded to auser device), run software remotely (e.g., hosting applications accessedby user devices), or otherwise provide applications support toapplications of user device 550. In some embodiments, information fromnetwork device 520 is provided to user device 550 using a client/serverapproach. For example, user device 550 may pull information from aserver, or a server may push information to user device 550. In someembodiments, an application client residing on user device 550 mayinitiate sessions with network device 520 to obtain information whenneeded (e.g., when data is out-of-date or when a user device receives arequest from the user to receive data). In some embodiments, informationmay include user information (e.g., user profile information,user-created content). For example, the user information may includecurrent and/or historical user activity information such as what contenttransactions the user engages in, searches the user has performed,content the user has consumed, whether the user interacts with a socialnetwork, any other suitable information, or any combination thereof Insome embodiments, the user information may identify patterns of a givenuser for a period of time. As illustrated, network device 520 includesentity information for a plurality of entities. Entity information 521,522, and 523 include metadata for the respective entities. Entities forwhich metadata is stored in network device 520 may be linked to eachother, may be referenced to each other, may be described by one or moretags in metadata, or a combination thereof.

In some embodiments, an application may be implemented on user device550, network device 520, or both. For example, the application may beimplemented as software or a set of executable instructions, which maybe stored in storage of the user device 550, network device 520, or bothand executed by control circuitry of the respective devices. In someembodiments, an application may include an audio recording application,a speech-to-text application, a text-to-speech application, avoice-recognition application, or a combination thereof, that isimplemented as a client/server-based application, where only a clientapplication resides on user device 550, and a server application resideson a remote server (e.g., network device 520). For example, anapplication may be implemented partially as a client application on userdevice 550 (e.g., by control circuitry of user device 550) and partiallyon a remote server as a server application running on control circuitryof the remote server (e.g., control circuitry of network device 520).When executed by control circuitry of the remote server, the applicationmay instruct the control circuitry to generate a display and transmitthe generated display to user device 550. The server application mayinstruct the control circuitry of the remote device to transmit data forstorage on user device 550. The client application may instruct controlcircuitry of the receiving user device to generate the applicationdisplays.

In some embodiments, the arrangement of system 500 is a cloud-basedarrangement. The cloud provides access to services, such as informationstorage, searching, messaging, or social networking services, amongother examples, as well as access to any content described above, foruser devices. Services can be provided in the cloud throughcloud-computing service providers, or through other providers of onlineservices. For example, the cloud-based services can include a storageservice, a sharing site, a social networking site, a search engine, orother services via which user-sourced content is distributed for viewingby others on connected devices. These cloud-based services may allow auser device to store information to the cloud and to receive informationfrom the cloud rather than storing information locally and accessinglocally stored information. Cloud resources may be accessed by a userdevice using, for example, a web browser, a messaging application, asocial media application, a desktop application, or a mobileapplication, and may include an audio recording application, aspeech-to-text application, a text-to-speech application, avoice-recognition application and/or any combination of accessapplications of the same. User device 550 may be a cloud client thatrelies on cloud computing for application delivery, or user device 550may have some functionality without access to cloud resources. Forexample, some applications running on user device 550 may be cloudapplications (e.g., applications delivered as a service over theInternet), while other applications may be stored and run on user device550. In some embodiments, user device 550 may receive information frommultiple cloud resources simultaneously.

In an illustrative example, a user may speak a voice query to userdevice 550. The voice query is recorded by an audio interface of userdevice 550, sampled and digitized by application 560, and converted to atext query by application 560. Application 560 may also includepronunciation along with the text query. For example, one or more wordsof the text query may be represented by phonetic symbols rather than aproper spelling. In a further example, pronunciation metadata may bestored with the text query, including a phonetic representation of oneor more words of the text query. In some embodiments, application 560transmits the text query and any suitable pronunciation information tonetwork device 520 for searching among a database of entities, content,metadata, or a combination thereof. Network device 520 may identify anentity associated with the text query, content associated with the textquery, or both and provide that information to user device 550.

For example, the user may speak “Show me Tom Cruise movies please” to amicrophone of user device 550. Application 560 may generate a text query“Tom Cruise movies” and transmit the text query to network device 520.Network device 520 may identify entity “Tom Cruise” and then identifymovies linked to the entity. Network device 520 may then transmitcontent (e.g., video files, trailers, or clips), content identifiers(e.g., movie titles and images), content addresses (e.g., URLs,websites, or IP addresses), any other suitable information, or anycombination thereof to user device 550. Because the pronunciations of“Tom” and “Cruise” are generally not ambiguous, application 560 need notgenerate pronunciation information in this circumstance.

In a further example, the user may speak “Show me the interview withLouis” to a microphone of user device 550, wherein the user pronouncesthe name Louis as “loo-ee” rather than “loo-ihs.” In some embodiments,application 560 may generate a text query “interview with Louis” andtransmit the text query to network device 520, along with metadata thatincludes a phonetic representation as “loo-ee.” In some embodiments,application 560 may generate a text query “interview with Loo-ee” andtransmit the text query to network device 520, wherein the text queryitself includes the pronunciation information (e.g., a phoneticrepresentation in this example). Because the name Louis is common, theremay be many entities that include this identifier. In some embodiments,network device 520 may identify entities having metadata that includes apronunciation tag having “loo-ee” as a phonetic representation. In someembodiments, network device 520 may retrieve trending searches, searchhistory of the user, or other contextual information to identify whichentity the user is likely referring to. For example, the user may havesearched “FBI” previously, and the entity Louis Freeh (e.g., formerdirector of the FBI) may include metadata that includes a tag for “FBI.”Once the entity is identified, network device 520 may then transmitcontent (e.g., video files or clips of interviews), content identifiers(e.g., file titles and still images from interviews), content addresses(e.g., URL, website, or IP addresses to stream one or more video filesof interviews), any other suitable information related to Louis Freeh,or any combination thereof to user device 550. Because the pronunciationof “Louis” may be ambiguous, application 560 may generate pronunciationinformation in such circumstances.

In an illustrative example, a user may speak “William Djoko” to amicrophone of user device 550. Application 560 may generate a textquery, which may not correspond to the correct spelling of the entity.For example, the voice query “William Djoko” may be converted to text as“William gjoka.” This incorrect text translation may result indifficulty in identifying the correct entity. In some embodiments,metadata associated with entity William Djoko includes alternativerepresentations based on pronunciation. The metadata for entity “WilliamDjoko” may include pronunciation tags (e.g., “related phrases”) as shownin Table 1.

TABLE 1 Illustrative metadata for an entity, including alternativerepresentations. ID: MN0001001577 TYPE: Music artist TITLE: WilliamDjoko RELATED PHRASES: William Kouam Djoko, William Cohan Joker, Williamgjoka, William joko PRONUNCIATION: Wilj 

 m Jō-kō LINKS: { }Because the text query may include an incorrect spelling, but themetadata associated with the correct entity includes variations, thecorrect entity may be identified. Accordingly, network device 520 mayinclude entity information including alternate representations, and thusmay identify the correct entity in response to a text query includingthe phrase “William gjoka.” Once the entity is identified, networkdevice 520 may then transmit content (e.g., audio or video files clips),content identifiers (e.g., song or album titles and still images fromconcerts), content addresses (e.g., URL, website, or IP addresses tostream one or more audio files of music), any other suitable informationrelated to William Djoko, or any combination thereof to user device 550.Because the name “Djoko” may be incorrectly translated from speech,application 560 may generate pronunciation information for storage inmetadata in such circumstances to identify the correct entity.

In the illustrative example above, the reachability of entity WilliamDjoko is improved by storing alternative representations, especiallysince the ASR process may result in a grammatically incorrect textconversion of the entity name.

In an illustrative example, metadata may be generated based onpronunciation for later reference (e.g., by text query or other searchand retrieval processes), rather than in response to a user's voicequery. In some embodiments, network device 520, user device 550, or bothmay generate metadata based on pronunciation information. For example,user device 550 may receive user input of an alternative representationof an entity (e.g., based on previous search results or speech-to-textconversions). In some embodiments, network device 520, user device 550,or both may automatically generate metadata for an entity using atext-to-speech module and a speech-to-text module. For example,application 560 may identify a textual representation of an entity(e.g., a text string of the entity's name), and input the textualrepresentation to the text-to-speech module to generate an audio file.In some embodiments, the text-to-speech module includes one or moresettings or criteria with which the audio file is generated. Forexample, settings or criteria may include language (e.g., English,Spanish, Mandarin), accent (e.g., regional, or language-based), voice(e.g., a particular person's voice, a male voice, a female voice), speed(e.g., playback time of the relevant portion of the audio file),pronunciation (e.g., for multiple phonetic variations), any othersuitable settings or criterion, or any combination thereof. Application560 then inputs the audio file to a speech-to-text module to generate aresulting textual representation. If the resulting textualrepresentation is not identical to the original textual representation,then application 560 may store the resulting textual representation inmetadata associated with the entity. In some embodiments, application560 may repeat this process for various settings or criteria, thusgenerating various textual representations that may be stored in themetadata. The resulting metadata includes the original textualrepresentation along with variations generated using text-speech-textconversions to forecast likely variations. Accordingly, when application560 receives a voice query from a user, and the translation to text doesnot exactly match an entity identifier, application 560 may stillidentify the correct entity. Further, application 560 need not analyzethe text query for pronunciation information, as the metadata includesvariations (e.g., analysis is performed upfront rather than in realtime).

Application 560 may include any suitable functionality such as, forexample, audio recording, speech recognition, speech-to-text conversion,text-to-speech conversion, query generation, search enginefunctionality, content retrieval, display generation, contentpresentation, metadata generation, database functionality, or acombination thereof In some embodiments, aspects of application 560 areimplemented across more than one device. In some embodiments,application 560 is implemented on a single device. For example, entityinformation 521, 522, and 523 may be stored in memory storage of userdevice 550, and may be accessed by application 560.

FIG. 6 shows a flowchart of illustrative process 600 for responding to avoice query based on pronunciation information, in accordance with someembodiments of the present disclosure. For example, a query applicationmay perform process 600, implemented on any suitable hardware such asuser device 400 of FIG. 4, user equipment system 401 of FIG. 4, userdevice 550 of FIG. 5, network device 520 of FIG. 5, any other suitabledevice, or any combination thereof. In a further example, the queryapplication may be an instance of application 560 of FIG. 5.

At step 602, the query application receives a voice query. In someembodiments, an audio interface (e.g., audio equipment 414, user inputinterface 410, or a combination thereof) may include a microphone orother sensor that receives audio input and generates an electronicsignal. In some embodiments, the audio input is received at an analogsensor, which provides an analog signal that is conditioned, sampled,and digitized to generate an audio file. The audio file may then beanalyzed by the query application at steps 604 and 606. In someembodiments, the audio file is stored in memory (e.g., storage 408). Insome embodiments, the query application includes a user interface (e.g.,user input interface 410), which allows a user to record, play back,alter, crop, visualize, or otherwise manage audio recording. Forexample, in some embodiments, the audio interface is always configuredto receive audio input. In a further example, in some embodiments, theaudio interface is configured to receive audio input when a userprovides an indication to a user input interface (e.g., by selecting asoft button on a touchscreen to begin audio recording). In a furtherexample, in some embodiments, the audio interface is configured toreceive audio input and begins recording when speech or other suitableaudio signals are detected. The query application may include anysuitable conditioning software or hardware for converting audio input toa stored audio file. For example, the query application may apply one ormore filters (e.g., low-pass, high-pass, notch filters, or band-passfilters), amplifiers, decimators, or other conditionings to generate theaudio file. In a further example, the query application may apply anysuitable processing to a conditioned signal to generate an audio filesuch as compression, transformation (e.g., spectral transformation,wavelet transformation), normalization, equalization, truncation (e.g.,in a time or spectral domain), any other suitable processing, or anycombination thereof. In some embodiments, at step 602, the controlcircuitry receives an audio file from a separate application, a separatemodule of the query application, based on a user input, or anycombination thereof. For example, at step 602, the control circuitry mayreceive a voice query as an audio file stored in storage (e.g., storage408), for further processing (e.g., steps 604-612 of process 600).

At step 604, the query application extracts one or more keywords fromthe voice query of step 602. In some embodiments, the one or morekeywords may represent the full voice query. In some embodiments, theone or more keywords include only important words or parts of speech.For example, in some embodiments, the query application may identifywords in speech, and select some of those words as keywords. Forexample, the query application may identify words, and among those wordsselect words that are not prepositions. In a further example, the queryapplication may identify as a keyword only a word that is at least threecharacters long. In a further example, the query application mayidentify keywords as a phrase including two or more words (e.g., to bemore descriptive and provide more context), which may be helpful tonarrow a potential search field of relevant content. In someembodiments, the query application identifies keywords such as, forexample, words, phrases, names, places, channels, media asset titles, orother keywords, using any suitable criteria to identify keywords from anaudio input. The query application may process words using any suitableword detection technique, speech detection technique, patternrecognition technique, signal processing technique, or any combinationthereof. For example, the query application may compare a series ofsignal templates to a portion of an audio signal to find whether a matchexists (e.g., whether a particular word is included in the audiosignal). In a further example, the query application may apply alearning technique to better recognize words in voice queries. Forexample, the query application may gather feedback from a user on aplurality of requested content items in the context of a plurality ofqueries, and accordingly use past data as a training set for makingrecommendations and retrieving content. In some embodiments, the queryapplication may store snippets (i.e., clips of short duration) ofrecorded audio during detected speech, and process the snippets. In someembodiments, the query application stores relatively large segments ofspeech (e.g., more than 10 seconds) as an audio file, and processes thefile. In some embodiments, the query application may process speech todetect words by using a continuous computation. For example, a wavelettransform may be performed on speech in real time, providing acontinuous, if slightly time-lagged, computation of speech patterns(e.g., which could be compared to a reference to identify words). Insome embodiments, the query application may detect words, as well aswhich user uttered the words (e.g., voice recognition) in accordancewith the present disclosure.

In some embodiments, at step 604, the query application adds detectedwords to a list of words detected in the query. In some embodiments, thequery application may store these detected words in memory. For example,the query application may store in memory words as a collection of ASCIIcharacters (i.e., 8-bit code), a pattern (e.g., indicating a speechsignal reference used to match the word), an identifier (e.g., a codefor a word), a string, any other datatype, or any combination thereof.In some embodiments, the media guidance application may add words tomemory as they are detected. For example, the media guidance applicationmay append a string of previously detected words with a newly detectedword, add a newly detected word to a cell array of previously detectwords (e.g., increase the cell array size by one), create a new variablecorresponding to the newly detected word, create a new filecorresponding to the newly created word, or otherwise store one or morewords detected at step 604.

At step 606, the query application determines pronunciation informationfor the one or more keywords of step 604. In some embodiments,pronunciation information includes a phonetic representation (e.g.,using the international phonetic alphabet) of the one or more keywords.In some embodiments, pronunciation information includes one or morealternative spellings of the one or more keywords to incorporate thepronunciation. In some embodiments, at step 606, the control circuitrygenerates metadata associated with the text query that includes aphonetic representation.

At step 608, the query application generates a text query based on theone or more keywords of step 604 and the pronunciation information ofstep 606. The query application may generate the text query by arrangingthe one or more keywords in a suitable order (e.g., in the orderspoken). In some embodiments, the query application may omit one or morewords of the voice query (e.g., short words, prepositions, or any otherwords determined to be relatively less important). The text query may begenerated and stored in suitable storage (e.g., storage 408) as a file(e.g., a text file).

At step 610, the query application identifies an entity among aplurality of entities of a database based on the text query and storedmetadata for the entity. In some embodiments, the metadata includes apronunciation tag. In some embodiments, the query application mayidentify the entity by identifying a metadata tag of a content item thatcorresponds to an entity. For example, a content item may include amovie having a tag for an actor in the movie. If the text query includesthe actor, then the query application may determine a match and mayidentify the entity as being associated with the content item based onthe match. To illustrate, the query application may identify the entityfirst (e.g., search among entities), and then retrieve contentassociated with the entity, or the query application may identifycontent first (e.g., search among content) and determine whether theentity associated with the content matches the text query. Databasesthat are arranged by entity, content, or both may be searched by thequery application.

In some embodiments, the query application identifies the entity basedon user profile information. For example, the query application mayidentify the entity based on a previously identified entity from aprevious voice query. In a further example, the query application mayidentify the entity based on popularity information associated with theentity (e.g., based on searches for a plurality of users). In someembodiments, the query application identifies the entity based on auser's preferences. For example, if one or more keywords match apreferred entity name or identifier of the user profile information,then the query application may identify that entity or more heavilyweigh that entity.

In some embodiments, the query application identifies the entity byidentifying the plurality of entities (e.g., with metadata stored foreach entity), determining a respective score for each respective entityof the plurality of entities based on comparing the respectivepronunciation tag with the text query, and selecting the entity bydetermining a maximum score. The score may be based on how many matchesare identified between keywords of the text query and metadataassociated with the entity or content item.

In some embodiments, the query application identifies more than oneentity (e.g., an associated metadata) among the plurality of entitiesbased on the text query. The query application may identify a contentitem that is associated with some or all of the entities of the query.In some embodiments, the query application identifies the entity bycomparing at least a portion of the text query to tags of the storedmetadata for each entity to identify a match.

At step 612, the query application retrieves a content item associatedwith the entity. In some embodiments, the query application identifies acontent item, downloads a content item, streams a content item,generates for display a content item, or a combination thereof. Forexample, a voice query may include “Show me recent Tom Cruise movies”and the query application may provide a link to the move “MissionImpossible: Fallout” that the user can select to view the video content.In some embodiments, the query application may retrieve a plurality ofcontent that is associated with entities matching the text query. Forexample, the query application may retrieve a plurality of links, videofiles, audio files, or other content, or a list of identified contentitems, in accordance with the present disclosure.

FIG. 7 shows a flowchart of illustrative process 700 for responding to avoice query based on alternate representations, in accordance with someembodiments of the present disclosure. For example, a query applicationmay perform process 700, implemented on any suitable hardware such asuser device 400 of FIG. 4, user equipment system 401 of FIG. 4, userdevice 550 of FIG. 5, network device 520 of FIG. 5, any other suitabledevice, or any combination thereof. In a further example, the queryapplication may be an instance of application 560 of FIG. 5.

At step 702, the query application receives a voice query. In someembodiments, an audio interface (e.g., audio equipment 414, user inputinterface 410, or a combination thereof) may include a microphone orother sensor that receives audio input and generates an electronicsignal. In some embodiments, the audio input is received at an analogsensor, which provides an analog signal that is conditioned, sampled,and digitized to generate an audio file. The audio file may then beanalyzed by the query application at steps 704. In some embodiments, theaudio file is stored in memory (e.g., storage 408). In some embodiments,the query application includes a user interface (e.g., user inputinterface 410), which allows a user to record, play back, alter, crop,visualize, or otherwise manage audio recording. For example, in someembodiments, the audio interface is always configured to receive audioinput. In a further example, in some embodiments, the audio interface isconfigured to receive audio input when a user provides an indication toa user interface (e.g., by selecting a soft button on a touchscreen tobegin audio recording). In a further example, in some embodiments, theaudio interface is configured to receive audio input, and beginsrecording when speech or other suitable audio signals are detected. Thequery application may include any suitable conditioning software orhardware for converting audio input to a stored audio file. For example,the query application may apply one or more filters (e.g., low-pass,high-pass, notch filters, or band-pass filters), amplifiers, decimators,or other conditionings to generate the audio file. In a further example,the query application may apply any suitable processing to a conditionedsignal to generate an audio file such as compression, transformation(e.g., spectral transformation, wavelet transformation), normalization,equalization, truncation (e.g., in a time or spectral domain), any othersuitable processing, or any combination thereof. In some embodiments, atstep 702, the control circuitry receives an audio file from a separateapplication, a separate module of the query application, based on a userinput, or any combination thereof. For example, step 702 may includereceiving a voice query as an audio file stored in storage (e.g.,storage 408), for further processing (e.g., steps 704-710 of process700).

At step 704, the query application extracts one or more keywords fromthe voice query of step 702. In some embodiments, the one or morekeywords may represent the full voice query. In some embodiments, theone or more keywords include only important words or parts of speech.For example, in some embodiments, the query application may identifywords in speech, and select some of those words as keywords. Forexample, the query application may identify words, and among those wordsselect words that are not prepositions. In a further example, the queryapplication may identify as a keyword only a word that is at least threecharacters long. In a further example, the query application mayidentify keywords as a phrase including two or more words (e.g., to bemore descriptive and provide more context), which may be helpful tonarrow a potential search field of relevant content. In someembodiments, the query application identifies keywords such as, forexample, words, phrases, names, places, channels, media asset titles, orother keywords, using any suitable criteria to identify keywords from anaudio input. The query application may process words using any suitableword detection technique, speech detection technique, patternrecognition technique, signal processing technique, or any combinationthereof. For example, the query application may compare a series ofsignal templates to a portion of an audio signal to find whether a matchexists (e.g., whether a particular word is included in the audiosignal). In a further example, the query application may apply alearning technique to better recognize words in voice queries. Forexample, the query application may gather feedback from a user on aplurality of requested content items in the context of a plurality ofqueries, and accordingly use past data as a training set for makingrecommendations and retrieving content. In some embodiments, the queryapplication may store snippets (i.e., clips with short durations) ofrecorded audio during detected speech, and process the snippets. In someembodiments, the query application stores relatively large segments ofspeech (e.g., more than 10 seconds) as an audio file, and processes thefile. In some embodiments, the query application may process speech todetect words by using a continuous computation. For example, a wavelettransform may be performed on speech in real time, providing acontinuous, if slightly time-lagged, computation of speech patterns(e.g., which could be compared to a reference to identify words). Insome embodiments, the query application may detect words, as well aswhich user uttered the words (e.g., voice recognition) in accordancewith the present disclosure.

In some embodiments, at step 704, the query application adds detectedwords to a list of words detected in the query. In some embodiments, thequery application may store these detected words in memory. For example,the query application may store in memory words as a collection of ASCIIcharacters (i.e., 8-bit code), a pattern (e.g., indicating a speechsignal reference used to match the word), an identifier (e.g., a codefor a word), a string, any other datatype, or any combination thereof.In some embodiments, the media guidance application may add words tomemory as they are detected. For example, the media guidance applicationmay append a string of previously detected words with a newly detectedword, add a newly detected word to a cell array of previously detectwords (e.g., increase the cell array size by one), create a new variablecorresponding to the newly detected word, create a new filecorresponding to the newly created word, or otherwise store one or morewords detected at step 704.

At step 706, the query application generates a text query based on theone or more keywords of step 704. The query application may generate thetext query by arranging the one or more keywords in a suitable order(e.g., in the order spoken). In some embodiments, the query applicationmay omit one or more words of the voice query (e.g., short words,prepositions, or any other words determined to be relatively lessimportant). The text query may be generated and stored in suitablestorage (e.g., storage 408) as a file (e.g., a text file).

At step 708, the query application identifies an entity based on thetext query and metadata for the entity. The metadata includes alternatetext representations of the entity based on pronunciation. In someembodiments, the query application may identify the entity byidentifying a metadata tag of a content item that corresponds to analternate representation of an entity. For example, a content item mayinclude a movie having a tag for an actor in the movie, wherein the tagincludes an alternate spelling (e.g., derived from a system such assystem 300, or otherwise included in the metadata). If the text queryincludes the actor, then the query application may determine a match andmay identify the entity as being associated with the content item basedon the match. To illustrate, the query application may identify theentity first (e.g., search among entities), and then retrieve contentassociated with the entity, or the query application may identifycontent first (e.g., search among content) and determine whether theentity associated with the content matches the text query. Databasesthat are arranged by entity, content, or both may be searched by thequery application. The query application may determine a match when oneor more words of the text query match an alternative representation ofan entity (e.g., as stored in metadata associated with the entity).

In some embodiments, the query application identifies the entity basedon user profile information. For example, the query application mayidentify the entity based on a previously identified entity from aprevious voice query. In a further example, the query application mayidentify the entity based on popularity information associated with theentity (e.g., based on searches for a plurality of users). In someembodiments, the query application identifies the entity based on auser's preferences. For example, if one or more keywords match analternative representation of a preferred entity name or identifier ofthe user profile information, then the query application may identifythat entity or more heavily weigh that entity.

In some embodiments, the query application identifies the entity byidentifying the plurality of entities (e.g., with metadata stored foreach entity), determining a respective score for each respective entityof the plurality of entities based on comparing the respective metadatawith the text query, and selecting the entity by determining a maximumscore. The score may be based on how many matches are identified betweenkeywords of the text query and metadata associated with the entity orcontent item.

In some embodiments, the query application identifies more than oneentity (e.g., an associated metadata) among the plurality of entitiesbased on the text query. The query application may identify a contentitem that is associated with some or all of the entities of the query.In some embodiments, the query application identifies the entity bycomparing at least a portion of the text query to tags of the storedmetadata for each entity to identify a match.

At step 710, the query application retrieves a content item associatedwith the entity. In some embodiments, the query application identifies acontent item, downloads a content item, streams a content item,generates for display a content item, or a combination thereof. Forexample, a voice query may include “Show me recent Tom Cruise movies”and the query application may provide a link to the move “MissionImpossible: Fallout” that the user can select to view the video content.In some embodiments, the query application may retrieve a plurality ofcontent that is associated with entities matching the text query. Forexample, the query application may retrieve a plurality of links, videofiles, audio files, or other content, or a list of identified contentitems, in accordance with the present disclosure.

FIG. 8 shows a flowchart of illustrative process 800 for generatingmetadata for an entity based on pronunciation, in accordance with someembodiments of the present disclosure. For example, an application mayperform process 800, implemented on any suitable hardware such as userdevice 400 of FIG. 4, user equipment system 401 of FIG. 4, user device550 of FIG. 5, network device 520 of FIG. 5, any other suitable device,or any combination thereof. In a further example, the application may bean instance of application 560 of FIG. 5. In a further example, system300 of FIG. 3 may perform illustrative process 800.

At step 802, the application identifies an entity of a plurality ofentities for which information is stored. In some embodiments, theapplication selects an entity based on a predetermined order. Forexample, the application may select entities in alphabetical order toperform portions of process 800. In some embodiments, the applicationidentifies the entity when the metadata for the entity is created. Forexample, the application may identify the entity when the entity isadded to the database (e.g., database of entities). In some embodiments,the application identifies the entity when a search operationmisidentifies an entity, and accordingly, an alternative representationmay be desired to prevent further misidentifications. In someembodiments, the application identifies the entity based on user input.For example, the user may indicate to the application (e.g., at asuitable user interface) the entity based on an incorrect search result,an unreachable entity, or an observed error in the search results. Insome embodiments, the application need not identify an entity inresponse to an error in search results or a predetermined order. Forexample, the application may randomly select entities of an entitydatabase and proceed to step 804. In some embodiments, the applicationmay identify an entity based on the entity's popularity in searchqueries. For example, greater search efficacy may be achieved bydetermining alternative representations for more popular entities suchthat more search queries are correctly responded to. In a furtherexample, the application may identify less popular, or even obscure,entities to prevent unreachability of those entities since very fewsearch queries may specify these entities. The application may apply anysuitable criteria to determine which entity to identify. In someembodiments, the application may identify more than one entity at step802, and accordingly steps 804-810 may be performed for each entityidentified. In some embodiments, the application may identify a contentitem rather than, or in addition to, an entity. For example, theapplication may identify an entity such as a movie, and then identifyevery other important entity associated with that entity to undergosteps 804-810.

At step 804, the application generates an audio file based on a firsttext string and at least one speech criterion. The first text stringdescribes the entity identified at step 802. For example, as illustratedin FIG. 3, the application may include text-to-speech engine 310, whichmay be configured to generate an audio file. The application maygenerate an audio output from a speaker or other suitablesound-generating device that may be detected by a microphone or othersuitable detecting device. The application may apply one or moresettings or speech criteria in generating and outputting the audio file.For example, aspects of the generated “voice” may be tuned or otherwiseselected based on any suitable criteria. In some embodiments, the atleast one speech criterion includes a pronunciation setting (e.g., howone or more syllables, letter groups, or words are pronounced, or whichphonemes are to be used). In some embodiments, the at least one speechcriterion includes a language setting (e.g., specifying a language, anaccent, a regional accent, or other language information).

In an illustrative example, including a plurality of speech criteria,the application may generate a respective audio file based on a firsttext string and a respective speech criterion, generate a respectivesecond text string based on the respective audio file, compare therespective second text string to the first text string, and store (e.g.,in metadata associated with the entity) the respective second textstring if it is not identical to the first text string.

In an illustrative example, the application may convert the first textstring to a first audio signal, generate speech at a speaker based onthe audio signal, detect the speech using a microphone to generate asecond audio signal, and process the audio signal to generate the audiofile. In some embodiments, the application generates the speech at thespeaker based on at least one speech setting of the text-to-speechmodule.

At step 806, the application generates a second text string based on theaudio file. The second text string should match the first text stringand describe the entity identified at step 802, aside from differencesthat may arise from the text-to-speech conversion or the speech-to-textconversion. For example, as illustrated in FIG. 3, the application mayinclude speech-to-text engine 320, which may be configured to receive anaudio input or file generated thereof and translate the audio to atranscript (e.g., a text string). The application may receive the audioinput at a microphone or other suitable sound-detecting device. Theapplication may apply one or more settings in receiving, conditioningand converting the audio file to text. For example, aspects ofconditioning and translating the detected “voice” may be tuned orotherwise selected based on any suitable criteria.

In an illustrative example, the application generates a playback of theaudio file at a speaker, detects the playback using a microphone togenerate an audio signal, and converts the audio signal to the secondtext string by identifying one or more words. In some embodiments, theapplication converts the audio signal to the second text string based onat least one text setting of the speech-to-text module.

At step 808, the application compares the second text string to thefirst text string. In some embodiments, the application compares eachcharacter of the first and second text strings to determine a match. Insome embodiments, the application determines to what extent the firsttext string and the second text string match (e.g., what fraction of thetext strings match, how many discrepancies exist, how many keywordsmatch or do not match). The application may use any suitable techniqueto determine if the first and second text strings are identical, similaror different, and to what extent they are similar or different.

At step 810, the application stores the second text string if it is notidentical to the first text string. In some embodiments, the applicationstores the second text string in metadata associated with the entity. Insome embodiments, step 810 includes the application updating existingmetadata based on one or more text queries. For example, as queries areresponded to and search results are evaluated, the application mayupdate the metadata to reflect the new learnings. If the second textstring is determined to be identical to the first text string, no newinformation is gained by storing the second text string. However, anindication of the comparison of step 808 may be stored in the metadatato increase confidence in the reachability of the entity via voicequeries. For example, if the second text string is identical to thefirst text string, this may serve to validate the existing metadata forvoice-based querying.

FIG. 9 shows a flowchart of illustrative process 900 for retrievingcontent associated with an entity of a voice query, in accordance withsome embodiments of the present disclosure. For example, a queryapplication may perform process 900, implemented on any suitablehardware such as user device 400 of FIG. 4, user equipment system 401 ofFIG. 4, user device 550 of FIG. 5, network device 520 of FIG. 5, anyother suitable device, or any combination thereof. In a further example,the query application may be an instance of application 560 of FIG. 5.

At step 902, the query application receives an audio signal at an audiointerface. The system may include a microphone or other audio detectingdevice and may record audio files based on audio input to the device.

At step 904, the query application parses the audio signal of step 902to identify speech. The query application may apply any suitabledecimation, conditioning (e.g., amplifying, filtering), processing(e.g., in the time or spectral domains), pattern recognition,algorithms, transforms, any other suitable actions, or any combinationthereof. In some embodiments, the query application identifies words,sounds, phrases, or a combination thereof using any suitable technique.

At step 906, the query application determines whether a voice query hasbeen received. In some embodiments, the query application determinesthat a voice query is received based on parameters of the audio signal.For example, periods without speech before and after the query maydelineate the extent of the voice query in a recording. In someembodiments, the query application identifies keywords in the spokenorder, and applies a sentence or query template to the keywords toextract a text query. For example, the arrangement of nouns, propernouns, verbs, adjectives, adverbs, and other parts of speech may providean indication of the beginning and end of the voice query. The queryapplication may apply any suitable criteria in parsing the audio signalto extract text. At step 908, the query application generates a textquery based on the results of steps 904, and 906. In some embodiments,at step 908, the query application may store the text query in suitablestorage (e.g., storage 408). If, at step 906, the query applicationdetermines that a voice query has not been received, or otherwise that atext query cannot be generated based on the parsed audio of step 904,the query application may return to step 902 and proceed to detect audiountil a voice query is received.

At step 910, the query application accesses a database for entityinformation. The query application uses the text query of step 908 tosearch among information of the database. The query application mayapply any suitable search algorithm to identify information, entities,or content of the database.

At step 912, the query application determines whether an entity of thedatabase of step 910 matches the text query of step 908. The queryapplication may identify and evaluate a plurality of entities to find amatch. In some embodiments, a text query includes more than one entity,and the query application searches among content to determine a contentitem that has the entities associated in metadata (e.g., by comparingthe text query to the metadata tags of the content item). In somecircumstances, the query application may be unable to identify a matchand in response may continue searching, search among another database,modify the text query (e.g., return to step 908, not shown), return tostep 904 to modify a setting used at step 904 (not shown), return anindication that no search results were found, any other suitableresponse, or any combination thereof. In some embodiments, the queryapplication may identify a plurality of entities, content, or both thatmatch the text query. Step 914 includes the query applicationidentifying content associated with the text query of step 908. In someembodiments, steps 914 and 910 may be reversed, and the queryapplication may search among content based on the text query. In someembodiments, the entity may include a content identifier, andaccordingly steps 910 and 914 may be combined.

At step 916, the query application retrieves content associated with thetext query of step 908. At step 916, for example, the query applicationmay identify a content item, download a content item, stream a contentitem, generate for display a content item or list of content items(e.g., or a list of links to content items), or a combination thereof.

The above-described embodiments of the present disclosure are presentedfor purposes of illustration and not of limitation, and the presentdisclosure is limited only by the claims that follow. Furthermore, itshould be noted that the features and limitations described in any oneembodiment may be applied to any other embodiment herein, and flowchartsor examples relating to one embodiment may be combined with any otherembodiment in a suitable manner, done in different orders, or done inparallel. In addition, the systems and methods described herein may beperformed in real time. It should also be noted, the systems and/ormethods described above may be applied to, or used in accordance with,other systems and/or methods.

1. A method for responding to voice queries, the method comprising:receiving a voice query at an audio interface; extracting, using controlcircuitry, one or more keywords from the voice query; determining, usingthe control circuitry, pronunciation information for the one or morekeywords; generating, using the control circuitry, a text query based onthe one or more keywords and the pronunciation information; identifyingan entity among a plurality of entities of a database based on the textquery and stored metadata for the entity, wherein the metadata comprisesa pronunciation tag; and retrieving a content item associated with theentity.
 2. The method of claim 1, wherein the pronunciation informationcomprises a phoneme of one of the one or more keywords.
 3. The method ofclaim 1, wherein identifying the entity is further based on user profileinformation.
 4. The method of claim 3, wherein the identifying theentity is based on a previously identified entity from a previous voicequery.
 5. The method of claim 1, wherein identifying the entity isfurther based on popularity information associated with the entity. 6.The method of claim 1, wherein identifying the entity comprises:identifying the plurality of entities, wherein respective metadata isstored for each entity of the plurality of entities, determining arespective score for each respective entity of the plurality of entitiesbased on comparing the respective pronunciation tag with the text query;and selecting the entity by determining a maximum score.
 7. The methodof claim 1, wherein the entity is a first entity, further comprisingidentifying a second entity among the plurality of entities based on thetext query and second metadata for the second entity, and wherein thecontent item is associated with the first entity and the second entity.8. The method of claim 1, wherein identifying the entity among aplurality of entities of the database comprises comparing at least aportion of the text query to tags of the stored metadata to identify amatch.
 9. The method of claim 1, wherein a first keyword of the one ormore keywords is associated with more than one pronunciation of thefirst keyword.
 10. The method of claim 1, wherein the pronunciationinformation comprises a phonetic representation of a first keyword ofthe one or more keywords.
 11. A system for responding to voice queries,the system comprising: an audio interface for receiving a voice query;and control circuitry coupled to the audio interface, the controlcircuitry configured to: extract one or more keywords from the voicequery; determine pronunciation information for the one or more keywords;generate a text query based on the one or more keywords and thepronunciation information; identify an entity among a plurality ofentities of a database based on the text query and stored metadata forthe entity, wherein the metadata comprises a pronunciation tag; andretrieve a content item associated with the entity.
 12. The system ofclaim 11, wherein the pronunciation information comprises a phoneme ofone of the one or more keywords.
 13. The system of claim 11, wherein thecontrol circuitry is further configured to identify the entity based onuser profile information.
 14. The system of claim 13, wherein thecontrol circuitry is further configured to identify the entity based ona previously identified entity from a previous voice query.
 15. Thesystem of claim 11, wherein the control circuitry is further configuredto identify the entity based on popularity information associated withthe entity.
 16. The system of claim 11, wherein the control circuitry isfurther configured to identify the entity by: identifying the pluralityof entities, wherein respective metadata is stored for each entity ofthe plurality of entities, determining a respective score for eachrespective entity of the plurality of entities based on comparing therespective pronunciation tag with the text query; and selecting theentity by determining a maximum score.
 17. The system of claim 11,wherein the entity is a first entity, wherein the control circuitry isfurther configured to identify a second entity among the plurality ofentities based on the text query and second metadata for the secondentity, and wherein the content item is associated with the first entityand the second entity.
 18. The system of claim 11, wherein the controlcircuitry is further configured to identify the entity among a pluralityof entities of the database by comparing at least a portion of the textquery to tags of the stored metadata to identify a match.
 19. The systemof claim 11, wherein a first keyword of the one or more keywords isassociated with more than one pronunciation of the first keyword. 20.The system of claim 11, wherein the pronunciation information comprisesa phonetic representation of a first keyword of the one or morekeywords. 21-50. (canceled)